Persist latency/error history + fix Overview GPU strip for remote nodes - #42
Merged
Conversation
Migration 017 adds request_metrics (one row per finalized request) and request_metrics_hourly (rollup). request_metrics.py provides record / record_from_trace (the tracer sink) and history/summary queries with exact per-bucket percentiles, optional group_by model|route. All SQL literal; grouping keys are chosen in Python (no identifier interpolation). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
RequestTracer gains an optional on_finalize callback invoked once per finalized request (the single chokepoint covering all terminal paths, dispatched and pre-dispatch). build_app wires it to request_metrics so every request's latency/status is persisted. Sink errors are caught and logged, never breaking request serving. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
request_metrics_rollup folds aged raw rows into request_metrics_hourly with exact per-bucket percentiles (all-models + per-model rows) and purges old hourly rows. history/summary read raw for short windows and the rollup for long windows. MetricsRollupTask (hourly) wired into app lifespan alongside UsageRollupTask. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Read-only latency/error history: per-bucket p50/p95 + error rate (raw for short windows, hourly rollup for long), optional group_by model|route, and a summary variant for the tiles/breakdown. Validation mirrors /admin/usage/series. api.ts gains getMetricsHistory/getMetricsSummary/getMetricsByModel + types. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Latency/errors tiles now read persisted history (/admin/metrics/history summary, 24h) and are badged "24h" instead of "live · 60s". Add a latency/errors over-time chart (p95 bars + error-rate strip) and a per-model latency/error breakdown table. Fix the GPU health strip to source GPUs from the cluster metrics snapshot (nodes[].gpus[]) grouped by node, so a GPU-less leader (VPS) shows remote worker-node GPUs; falls back to local topology only when the snapshot has none. Extract a shared GpuMeter used by both paths. Header GPU count now reflects the cluster total. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Latency and errors on the Overview dashboard were live-only (a ~60s rolling window that died on restart). This persists them so they gain real multi-day history — and fixes a related GPU display bug for cluster setups.
RequestTracer.finalizechokepoint writes onerequest_metricsrow per finalized request (migration 017). It captures all terminal outcomes, including pre-dispatch failures (bad model, auth, no-ready-service), so the error rate is complete. Sink errors are caught and logged, never breaking request serving.request_metrics_hourly) with exact per-bucket percentiles computed at rollup time, retainedretention_days.MetricsRollupTaskmirrorsUsageRollupTask.GET /admin/metrics/history(per-bucket p50/p95 + error rate, raw for short windows / rollup for long,group_by=model|route,summaryvariant). All SQL literal (Bandit-clean).24h), plus a latency/error over-time chart (p95 bars + error-rate strip) and a per-model latency/error breakdown.nodes[].gpus[]) grouped by node, so a GPU-less leader (VPS) shows remote worker-node GPUs; falls back to local topology only when the snapshot has none. Extracted a sharedGpuMeter.Design/plan:
docs/superpowers/specs/2026-06-05-metrics-persistence-design.md(local, gitignored).Test Plan
/admin/metrics/history).ruff+mypy+banditclean.tsc -b+vite buildgreen.request_metrics(dispatched + pre-dispatch errors, two models): history/summary/group_by endpoints correct (400 dispatched + 12 pre-dispatch captured; per-model p95 distinguishes a slow model); Overview tiles show 24h numbers, chart + breakdown render, GPU strip renders from the snapshot. Zero browser console errors.🤖 Generated with Claude Code